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ON AN ADAPTIVE REGULARIZATION FOR ILL-POSED 
NONLINEAR SYSTEMS AND ITS TRUST-REGION 
IMPLEMENTATION * 

S. BELLAVIAtAND B. MORINitAND E. RICCIETTI t 


Abstract. In this paper we address the stable numerical solution of nonlinear ill-posed systems 
by a trust-region method. We show that an appropriate choice of the trust-region radius gives 
rise to a procedure that has the potential to approach a solution of the unperturbed system. This 
regularizing property is shown theoretically and validated numerically. 

Keywords: Ill-posed systems of nonlinear equations, regularization, nonlinear stepsize 
control, trust-region methods. 

1. Introduction. Nonlinear systems modeling inverse problems are typically 
ill-posed, in the sense that their solutions do not depend continuously on the data 
and their data are affected by noise [maiii]. In this work we focus on the stable 
approximation of a solution of these problems. Procedures in the classes of Levenberg- 
Marquardt and trust-region methods are discussed, and a suitable version of trust- 
region algorithm is shown to have regularizing properties both theoretically and nu¬ 
merically. The underlying motivation for our study is twofold: most of the practical 
methods in literature have been designed for well-posed systems, see e.g., [am], and 
thus are unsuited in the context of inverse problems; adaptation of existing procedures 
for handling ill-posed problems, carried out in the seminal papers [iiiniiiiiiiiiiijiia, 
deserves further theoretical and numerical insights. 

Let 


F{x)=y, (1.1) 

with F : R" —>■ R" continuously differentiable, be obtained from the discretization 
of a problem modeling an inverse problem. It is realistic to have noisy data at 
disposal, satisfying 


\\y-y^\\2<S, ( 1 . 2 ) 

for some positive <5. Thus, in practice it is necessary to solve a problem of the form 

F{x) = y\ (1.3) 

and, due to ill-posedeness, possible solutions may be arbitrarily far from those of the 
original problem. 

In [101 [TT| . Hanke supposed that an initial guess, close enough to some solution 
x'' of (11.11) . is available. Then, he proposed a regularizing Levenberg-Marquardt pro¬ 
cedure which is able to compute a stable approximation to x^ or to some other 
solution of the unperturbed problem dnD close to x'^. This task is achieved through 
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a nonlinear stepsize control in the Levenberg-Marquardt procedure and the discrep¬ 
ancy principle as the stopping criterion, so that the iterative process is stopped at the 
iteration fc* satisfying 

\\y^-F{xl)h<TS<\\y^-F{xi)h, 0 < k < K, (1.4) 

with r > 1 appropriately chosen [TB] . Remarkably xf.^ converges to a solution of dm 
as S tends to zero. 

Further regularizing iterative methods have been proposed, including first-order 
methods and Newton-type methods. Analogously to the Levenberg-Marquardt pro¬ 
cedure proposed by Hanke, instead of promoting convergence to a solution of (jl.3ll . 
they form approximations of increasing accuracy to some solution of the unperturbed 
problem (HI) until the discrepancy principle (11.41) is met. We refer to mm for the 
description and analysis of such methods. 

The above mentioned regularizing Levenberg-Marquardt method belongs to the 
unifying framework of nonlinear stepsize control algorithms for unconstrained opti¬ 
mization developed by Toint [20] and including trust-region methods [3]. Therefore, 
elaborating on original ideas by Hanke, we introduce and analyze a regularizing vari¬ 
ant of the trust-region method. The main feature of our variant is the rule for selecting 
the trust-region radius which guarantees two properties. First, it guarantees the same 
regularizing properties as the method by Hanke. Second, as for standard trust-region 
procedure, it enforces a monotonic decrease of the value of the function 

Hx) = l\\y'-F{x)\\l (1.5) 

at the iterates xf,. Convergence properties are enhanced with respect to the regu¬ 
larizing Levenberg-Marquardt procedure in the following respects. With exact data, 
if there exists an accumulation point of the iterates which solves (HI), then any ac¬ 
cumulation point of the sequence solves (HD- With noisy data, the methods have 
the potential to satisfy the discrepancy principle (HD- As for standard trust-region 
methods, these properties can be enhanced independently of the closeness of the initial 
guess to a solution of m- 

Our contribution covers theoretical and practical aspects of the method proposed. 
From a theoretical point of view, we propose the use of a trust-region radius converging 
to zero as <5 tends to zero. Trust-region methods with this distinguishing feature have 
been proposed in several papers, see [3 El H [25], but none of such works was either 
devised for ill-posed problems or applied to them; thus, our study offers new insights on 
the potential of this choice for the trust-region radius. Moreover, we have made a first 
attempt toward global convergent methods for ill-posed problems; to our knowledge, 
this topic has been considered only in a multilevel approach proposed by Kaltenbacher 
m- Finally, local convergence analysis has been carried out without the assumption 
(commonly made in literature) on the boundness of the inverse of the Jacobian J of 
F, since it may not be fulfilled in the situation of ill-posedeness. Taking into account 
that the standard convergence analysis of trust-region methods always requires the 
invertibility of J, our results represent a progress in the theoretical investigation of 
convergence. Concerning numerical aspects, we discuss an implementation of the 
regularizing trust-region method, and test its ability to approximate a solution of 
(HJ in presence of noise. Comparison with a standard trust-region scheme highlights 
the impact of the proposed trust-region radius choice on regularization. 

The paper is organized as follows. In |2] we describe the main features of the 
regularizing Levenberg-Marquardt method proposed by Hanke. In ([3] we introduce 
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our regularizing version of the trust-region methods and in 2] we study the local con¬ 
vergence properties. A comparative numerical analysis of all the procedures studied 
is done in IJS] 

Notations. We indicate the iterates of the procedures analyzed as if the 
data are exact, Xk may be used in alternative to By a;g = xq we denote an initial 
guess which may incorporate a-priori knowledge of an exact solution. The symbol || • || 
indicates the Euclidean norm. The Jacobian matrix of F is denoted as J. 

2. Regularizing Levenberg-Marquardt method for ill-posed problems. 

We describe the regularizing version of the Levenberg-Marquardt method proposed 
in m for solving (IE3I, and analyze some issues for its practical implementation. 

At fc-th iteration of the Levenberg-Marquardt, given xf. £ R" and Xk > 0, let 

= ^l|A(4) - / + Jixi)pf + ^Afcllpf, (2.1) 

be a quadratic model around xf. for the function $ in (II.5L see mill]. The step pk 
taken minimizes and xf.^-^ = x^. +pk- We observe that, if p{X) is the solution of 

{Bk + XI)piX) = (2.2) 

with Bk = J{xlY' J{xl) and gk = J(xf )^(E(a;f) - y^), then pk = p{Xk). 

If problem (11.31) is ill-posed, and the scalars Xk are limited to promote convergence 
of procedure, see (TH] , then the solution of (HHI) may be significantly misinterpreted 0 
[Ml [22]. The regularizing Levenberg-Marquardt method [lOj attempts to approximate 
solutions of dnD by choosing Xk as the solution of the nonlinear scalar equation 

\\F{xi) - / + J{xi)p{X)\\ = q\\F{xi) - /II, (2.3) 


for some fixed q £ (0,1). Under suitable assumptions discussed below, A®, is uniquely 
determined from ()2.3|1 . 

As for (12.3L it is useful to establish relations between A, ||p(A)|| and \\F{xl) — 

/ + Jixi)piX)\\. 

Lemma 2.1. |71 Lemma 4-^] Suppose ||gfe|| ^ 0 and let p{X) be the minimum 
norm solution of with A > 0. Suppose furthermore that J{x\) is of rank ^ 

and its singular-value decomposition is given by Uk"BkV^ where is the diagonal 
matrix with entries m, • •., “fn on the diagonal. Then, denoting r = (ri, r 2 ,..., r„)^ = 
^ki^i^k) ~y^)> '^6 have that 


lb(A)f = 


’7. 7. 




\\F{xi) - / + J{xi)p{X)f = ^ 

i—1 ^ ^ ^ 


E’ 

i=l+\ 


(2.4) 

(2.5) 


Using this result, the solution of (12.31) is characterized as follows. 

Lemma 2.2. Suppose ||gfc|| 0. Let p(X) be the minimum norm solution of S2.2\) 

with A > 0, Ti{J{xk))^ be the orthogonal complement of the range 'lZ[J{xjf)) of J{xjf), 
and be the orthogonal projector onto TZ[J{x^ff})'^. Then 
(i) Equation i2.S\) is not solvable */|//^"(®fe) — 2/^)|| > ~?/^ll- 
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( 2 . 6 ) 


(^^) If 


11^(4) - 4 + - 4)11 < |-ll^(4) - 4ll: 

for some 6^ > 1; then equation \2.3\) has a unique solution such that 


Proof, (i) Equation (j2.5l) implies 

lim ||F(4) -/ + J{xi)p{\)\\ = \\P^{F{xi) - /)||, 

A—>-0 

lim ||F(4) - 4 + J{xi)p{X)\\ = \\F{xi) - 4||. 

A^oo 


(2.7) 


Thus, since ||E(x^) — + J(a;^,)p(A)|| is monotonically increasing as a function of A, 

we conclude that (12.31) does not admit solution if WP^iF^xf.) — 2 /'^)|| > q\\F{x^) —y^\\. 

(a) Trivially ||4(^(4) “4)11 < 14(4) “4+ '^(4)4“4)ll> for any X. Hence, 

for the monotonicity of |4(a;^,) — y^ + J(a;^,)p(A)||, if (12.61) holds, then equation (12.3p 
admits a solution which is positive and unique. Finally, 

Fixi) - 4 + Ji4)pW = KJ{xi)J{xif + \I)-\F{xi) - 4), 


see e.g., (TUI Proposition 2.1], and by (12.31) 

<?IIf(4) - 4|| = a«ii(j(4) J(4)^ + a^4)''(^(4) - 4) 
> .. |4(4)-4lh 


\\Ji4W + 4' 


which yields (1^ . 0 

In [10) . the analysis of resulting Levenberg-Marquardt method was made under 
the subsequent assumptions on the solvability of the problem dm, the Taylor re¬ 
mainder of F, and the vicinity of the initial guess xq to some solution x^ of (HID. 


Assumption 2.1. Given an initial guess xq, there exist positive p and c such 
that system a is solvable in Bp{xo), and 

\\F{x) - F{x) - J{x){x - x)|| < c||x - i|| |4(x) - E(i)||, x,x G i? 2 p(xo). (2.8) 


Assumption 2.2. Let Xo, c and p as in Assumvtion \2.1l x^ he a solution of EH) 
and Xo satisfy 

||xo -xf|| < min|^,p| , 

||xo-xf|| <min|4I^,p|, 

where t > 1/q. 

From Assumption 1 2.1 1 it follows that inequality (12.61) is satished for any x| belong¬ 
ing to i? 2 p(xo) and consequently there exists a solution to (12. 3L see [TUI Theorems 
2.2, 2.3]. 


if 6 = 0, (2.9) 

if (5 > 0 (2.10) 
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Under Assumptions l2.1l and l2.21 the approximations xf., generated by the Levenberg-| 
Marquardt method satisfy (HH) and converge to a solution of (ED as S tends to zero. 

Theorem 2.3. Let Assumptions [K7\ and 1 2. 2\ hold and x^. be the Levenberg- 
Marquardt iterates determined by using S2.S\) . For noisy data, suppose k < k„ where 
fc* is defined in Pl\ >- Then, any iterate x^. belongs to B 2 p{xo). With exact data, 
the sequence converges to a solution of m.lM . With noisy data, the stopping 

criterion o is satisfied after a finite number fc* of iterations and } converges 
to a solution of fzny as S goes to zero. 

Proof See [TU], Theorem 2.2 and Theorem 2.3. □ 

Let us focus on a specific issue concerning the implementation of the method 
which, to our knowledge, has not been addressed either in m or in related papers. 
The numerical solution of (|2.3I) requires the application of a root-finder method and 
Newton method is the most efficient procedure, though in general it requires the 
knowledge of an accurate approximation to the solution. On the other hand, nonlinear 
equations which are monotone and convex (or concave) on some interval containing 
the root are particularly suited to an application of Newton method, see e.g. [H 
Theorem 4.8]. Equation (12.31) does not have such properties but it can be replaced 
by an equivalent equation with strictly decreasing and concave function in [A^,oo); 
thus, Newton method applied to the reformulated equation converges globally to 
whenever the initial guess overestimates such a root. 

Lemma 2.4. Suppose ||F(a:^) — y'^j] 7^ 0, and that S2.A} has positive solution A^. 

Let 

-y^ + Ji^DpWW -y^ll 

Then, Newton method applied to S2.11\) converges monotonically and globally to the 
root A| of (2. SI) for any initial guess in the interval [A|, 00 ). 

Proof. Trivially, solving (12.3p is equivalent to finding the positive root of the 
equation (12.1111 . We now show that ip{X) is strictly decreasing in [A^, 00 ) and concave 
on (0, 00 ). By (12.511 . 


_A_ 

\\F{xi) - y^ + J{xi)p{X)\\ 



( 2 . 12 ) 


and this function is concave on (0,oo), cfr. [H Lemma 2.1]. Thus, ip is concave on 
(0,oo) and trivially ip'{X) is strictly decreasing. 

Now we show that ip'{X\) is negative; thus, using the monotonicity of ip'{X), we 
get that ip{X) is strictly decreasing in [A^,,oo). Differentiation of ip{X) and (12.31) give 




__ 

\\F{,xl) - + J{xi)p{XlW 

m r^K , V \\n4)-y^ + Ji4)piK)f 

\\F{xl)-y^+ J{xl)p{Xl)\\^ + Xl)^ .^^\XlJ (A«)2 


E 


(C 




n 2 


<}\\F{xi)-y^\ 
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Moreover, using (I2.12L it holds 


^'(A|) = 




WFixi) -y^ + Jixi)pixiw (cf + Kr ^ Vcf + 


E' 


rUl 


-E 


(AD^ rid 


WFixi) -y^ + j(4MADP i(? + Kr ’ 


E 


i.e. ' 0 '(A^) is negative. 

The claimed convergence of Newton method follows from results on univariate 
concave functions given in pi Theorem 4.8]. □ 

For the practical evaluation of V^(A) and tp'{X) we refer to ITT]. 

Since (12.31) may have not solution, Hanke observed that it may be replaced by 

||i^(4) -y^ + JixDpkW > q\\Fixi) - 2 /^ 11 , (2.13) 

later denoted as the q-condition, PI Remark p. 6 ] but this criterion was not analyzed 
or employed in numerical computation. Since our aim is to tune Xk in view of global 
convergence, while preserving regularizing properties, in the next section we allow 
more flexibility in its selection and design a trust-region method based on condition 

(IM) . 

3. A regularizing trust-region method. Trust-region methods are globally 
convergent approaches where the stepsize between two successive iterates is deter¬ 
mined via a nonlinear control mechanism [3] . At a generic iteration fc of a trust-region 
method, the step pk solves 

minmj^ip) = i||F(xi) “ / + : (3 

s.t. IIpII < Afe, 

where Ak is a given positive trust-region radius. If ||( 7 fe|| ^ 0 then pk solves (13.ip if 
and only if it satisfies ( 12 . 21 ) for some nonnegative Xk such that 

Afe(|bfe||-Afe) = 0. (3.2) 

Therefore, whenever the minimum norm solution p'^ of 

BkP = -gk, 

satisfies ||p+|| < Ak, then the scalar Xk is null and pk = p{0) solves (13.11) . Otherwise, 
the step Pk = p(Afc) is a Levenberg-Marquardt step. If ||pfe|| = Ak, then the trust- 
region is said to be active. 

Starting from an arbitrary initial guess, trust-region methods generate a sequence 
of iterates such that the value of $ in (11.51) is monotonically decreasing and this feature 
is enforced by an adaptive choice of the radius A^. Specifically, let pk be the trust- 
region step and 


_ aredjpk) 
predipk) ’ 
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(3.3) 












be the ratio between the achieved ared{pk) and predicted pred{pk) reductions given 
by 

ared{pk) = $(4) “ ^(4 + Pfc). (3-4) 

pred{pk) = 4>(4) “ "r™(Pfe)- (3-5) 

Then, the trust region radius is reduced if pk is below some small positive threshold; 
otherwise it is left unchanged or enlarged [3]. 

Since trust-region steps and Levenberg-Marquardt steps have the same form (j2.2ll , 
trust-region and Levenberg-Marquardt methods fall into a single unifying framework 
which can be used for their description and theoretical analysis [illTlIlD]. Due to such 
a strict connection, we elaborate on the regularizing Levenberg-Marquardt described 
in the previous section, and introduce a regularizing variant of trust-region methods 
for solving ill-posed problems. 

The standard trust-region strategy is modihed so that the nonlinear stepsize con¬ 
trol enforces both the monotonic reduction of $ and the (/-condition (12.131) . To this 
end, we first characterize the parameters A such that p{X) satisfies (12.131) . 

Lemma 3.1. Assume ||(/fc|| 0. Let p{X) be the minimum norm solution of i2.2\} 

with A > 0 and be the orthogonal projector onto TZ{J. Then, equation h2.13\) 
is satisfied for any A > 0 whenever 

\\Pi{F{xi)-y^)\\>q\\F{xi)-/\\. (3.6) 

Otherwise, it is satisfied for any X > A^, where A^ satisfies 
Proof. The claims easily follow from Lemma [2.21 □ 

Now we are ready to characterize the size of the trust-region radius guaranteeing 

(E331). 

Lemma 3.2. Let p^ solve the trust-region problem mi- If 

(3.7) 

then Pk satisfies the q-condition \2.1S\) . 

Proof. By Lemma |3.1I we know that the g-condition is satisfied either for A > 0, 
or for any X > Xf. In the former case, the claim trivially holds. In the latter case, by 
(1^ it follows 


and by (12.7p it holds 


lb(ADII > 


llgfcll 

\\Bk + Xll\\ ’ 


\\Bk + XlI\\<^^. 

By construction ||pfc|| < A/,., and if (j3.7|) holds then we obtain 


\\Pk\\ 


lb(Afe)||<^lbfe||< 


Ibfcll 

\\Bk + Xll\\ 


< \\piK)l 


Since |b(A)|| is monotonically decreasing, it follows Xk > A^ and condition (|2.13l) is 
satisfied. □ 
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We stress that the bound (1X71) provides a practical rule for choosing the trust- 
region radius that guarantees the satisfaction of the q-condition (12.131) . Conversely, 
in papers |23| and |25j . where trust-region for ill-posed problems are studied, such a 
condition is respectively assumed to be satisfied and explicitely enforced rejecting the 
step whenever it does not hold. 

The result in Lemma [3.2l suggests the trust-region iteration described in Algorithm 
13.11 We distinguish between the parameters needed in the case of exact-data and the 
parameters required with noisy-data. 


Algorithm 3.1: fcth iteration of the regularizing Trust-Region method 
for problem (II.3p 

Given , rj € (0,1), 7 G (0,1), 0 < Cmin < Gmax- 
Exact-data: given (5 = 0, g G (0,1). 

Noisy-data: given (5 > 0, r > 1, g > l/r. 


1 . 

2 . 

3. 


4. 


Compute Bk 
Choose Afe G 


= J{xi) and gk = 

C^min II; niin ■< (7max; 


= j{xinF{xi) 


y^)- 


Repeat 

3.1 Compute the solution pk of the trust-region problem (13.111 . 

3.2 Compute pk given in (I3.3I) - (I3.5I) . 

3.3 If Pk < then set Afc = yAfe. 

Until Pk > g. 

Set =xl+pk. 


Due to well know properties of trust-region methods. Algorithms 13.11 is well- 
defined, i.e. if llgfcll 7 ^ 0 the step pk is found within a finite number of attempts, 
provided that the following Assumption is met [3]. 

Assumption 3.1. There exists a positive eonstant kj such that 

for any x belonging to the level set £ = {a; G M" s.t. $(a;) < <i>(a:o)}. 

Global convergence of the trust-region method is stated in the following theorem; 
we refer to [la Theorem 11.9] for the proof. 

Theorem 3.3. Suppose that Assumvtion \3.1\ holds and J is Lipschitz continuous 
on R". Then, the sequence {xf} generated by Alaorithm \3.1\ satisfies 

hm V$(4) = hm \\J{xif{Fixi) - y^)|| = 0. (3.8) 

K—¥(yO K—¥CC> 

We observe that assumption on Lipschitz continuity of J is made also in the 
paper m- By construction, the sequence {||G(a:f) — ?/'*||} is monotonically decreasing 
and bounded below by zero; hence it is convergent. Equation (13.8p implies that any 
accumulation point of the sequence {xf.} is a stationary point of $. As for exact data, 
we conclude that if there exists an accumulation point of {xk} solving (II. ip . then any 
accumulation point of the sequence solves dm. In the case of noisy data, if the value 









of $ at some accumulation point of {a;^} is below the scalar rJ, then there exists an 
iterate x^. such that the discrepancy principle is met. 

It remains to show the behaviour of the sequences generated by Algorithm 13.11 
when, for some k, xf. is sufficiently close to a solution x^ of (HI). For instance, this 
occurs with exact data when the accumulation points of {xfc} solve ( 11 . 11 ) and k is 
sufficiently large. In the next section we show that the trust-region method described 
in Algorithm 13.11 shares the same local regularizing properties as the regularizing 
Levenberg-Marquardt method. 

4. Local behaviour of the trust-region method. We analyze the local prop¬ 
erties of the trust-region method under the same assnmptions made for the Levenberg- 
Marquardt method. Hence, we suppose that there exists an iteration index k such 
that the iterate x^ satisfies the following assumptions that are the counterpart of 
Assumptions 12.11 and 12.21 for the Levenberg-Marquardt method. 

Assumption 4.1. Suppose that for some iteration index k there exist positive p 
and c such that system is solvable in Bp{x^), and 


||F(a;) - F{x) - J{x){x - x)|| < c\\x - x\\ ||F(x) - F(i)||, x,x & B 2 p{x\). (4.1) 


Moreover, letting x^ be a solution of o, and T > 1/q if the data are noisy, suppose 
that x^ satisfies 


< min|-,p| , 

*/ 

II 

o 

(4.2) 

1 C J 





(5 > 0 . 

(4.3) 


To our knowledge, except for paper [531 Hi], local convergence properties of trust- 
region strategies have been analyzed under assumptions which involve the inverse of 
J and its upper bound in a neighbourhood of a solution, and thus are stronger than 

m- 

The following theorems show the local behaviour of the regularizing trust-region 
method. We prove that locally the trust-region is active, the iterates xf. with k > k 
remain into the ball Bp(x^) and the resulting algorithm is regularizing. We remark 
that in standard trust-region methods, the trust-region becomes eventually inactive. 
On the other hand, regularization requires strictly positive scalars Afc, and conse¬ 
quently an active trust-region in all iterations. First we focus on the noise-free case 
and we show that the error \\xk — a;i|| decreases in a monotonic way for k > k, and 
the sequence {a;^} converges to a solution of ( 11 . 11 ) . 

Theorem 4.1. Suppose that Assumvtions rOl and \4.1\ hold and 5 = 0. Then, 
Alaorithm \3.1\ generates a sequence {xk} such that, for k> k, 

(i) the trust-region is active, i.e. Afc > 0 and Xk belongs to Bp{Xf.); 

(a) \\xk+i - x^l < \\xk - x'^ll; 

(Hi) there exists a constant A > 0 such that Afc < A. 

Moreover, 

(iv) the sequence {xk} converges to a solution of 

Proof (i)-(ii) The scalar A^ in Algorithm 13.11 is such that Xj. > A|. From 

Assumption 14.11 condition (12.61) is satisfied a.t k = k with Oj. = -r- - —r--, and 

clixfc -a;T|| 
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consequently by Lemma 12.21 is strictly positive. Hence, the trust-region is active. 
Further, by a straightforward adaptation of m Proposition 4. 10 it follows 

Ikfc - Ikfe+i ^'^ \\F{xk)-y + J{xk)Pk\\^, 

and this implies \\xj^^i ~ as (14.211 guarantees 0^ > 1. Consequently 

a^fc+i G Bp{xi^). Repeating the above arguments, by induction we can prove that 
condition (12.61) holds for k > k, Xk > 0, and the following inequality holds 

Wxk-x'^f - ||a;fe+i -x^W^ > ^\ \F{xk)-y + J{xk)pk\\'^, (4.4) 

CfcAfe 


with 


Ok 



> 1 . 


Thus, by induction, the sequence {||a:fc — is monotonic decreasing and the 

sequence is monotonic increasing. 

(in) Since the trust-region is active, by (12.21) 

Afe = \\Pk\\ = WiBk + Xkiy^gkW < (4.5) 

Afe 

Thus our claim follows if Afc/||( 7 fc|| is larger than a suitable threshold, independent 
from k. Let us provide such a bound by estimating the value of which guarantees 
condition pk > r]- If this condition is fulfilled for the value of A^ fixed in Step 2 of 
Algorithm 13.11 then Afc/||gfe|| > Cmin; otherwise, the trust-region radius is progres¬ 
sively reduced, and we provide a bound for the value of A^ at termination of Step 3 
of Algorithm 13.ll in the case where ^(xk+Pk) > ixik^ipk)- This occurrence represents 
the most adverse case; in fact if ^{xk + Pk) < ''TT-™(Pfc) then Pfc > 1 > ?7 and the 
repeat loop terminate for a trust-readius greater than or equal to the one estimated 
below. Trivially, 


. _ _ ^■(xfc +Pk)- mj^jpk) 

^{Xk) - TO™(pfc) 

and 

^{xk +Pk) - m™(pfe) < ]^\\F{xk + Pk) - F{xk) - J{xk)pk\\'^ 

+ \\F{xk+Pk) - F{xk) - J{xk)pk\\\\F{xk) - y + J{xk)pk\\- 
By (14.11) and the mean value [TH Theorem 11.1], it holds 

\\F{xk + Pk) - F{xk) - J{xk)pk\\ < c\\pk\\\\F{xk+Pk) - F{xk)\\ <cKj\\pk\f. (4.6) 
Consequently, as Afc < CmaxH^fclli 

"^{xk +Pk) -mj^ipk) < ic/cjA^||A(xo) -?/||(ck]}C'^,,,,,||F(xo) - y\\ +2). 


cfr. equation (4.6) in |14l Proposition 4.1] 


10 








Theorem 6.3.1 in [3] shows that 


<^{xk) - rnl^{pk) > min \ Ak, 


llgfcll 1 

\\Bk\\)- 


Then, 


whenever < 


llg.-ll 


^{xk) - TOfc bfc) > 2^k\\gk\l 


and this implies 


1 - Pfc < 


CKjAkWFjxo) - y\\{cK}Cl,^J\Fjxo) - y\\ + 2) 

llg^cll 


Namely, termination of the repeat loop occurs with 

Afe < llfffcllw, 


and 

1 1 ~ g 

kY ckj\\F{xo) - y\\{cK^jC^,^J\F{xo) - y\\ + 2) 

Taking into account Step 2 and the updating rule at Step 3.3, we can conclude that, 
at termination of Step 3, the trust-region radius Ak satisfies 




Afc > minlCmin, 7“^} llgfell- 


Finally, by (14.51) Afc < A as 


Afc < 


M 

Afc 


< max 


I 

JUl Cmin J 


(4.8) 


(iv) Since both the function {9 — 1)/9 and the sequence {^^fcj^fc are monotonic 
increasing, it follows {9k — 1)/9k > {9k — l)j9k and by (|4.4I) 


\\xk - xtf - ||xfc+i - xtf > ^^^^\\F{xk) - y + J{xk)pkr. (4.9) 

Therefore, following the lines of the proof of Theorem 4.2 in m, we can conclude 
that {xk} is a Cauchy sequence, i.e., it is convergent. 

Finally, by (14.91) . Afc < A and (I2.13P 

lixfc - xtf - ||xfc+i - xtf > ^i^L_l)^||F(xfc) - 2 /f, 

^fcA 

Hence ||F(xfc) — y\\ tends to zero and the limit of Xk has to be a solution of (11.11) . 

□ 

A similar result can be given for the noisy case. In the following theorem we prove 
that for fc < fc < fc*, where fc* is defined in (11.41) , the trust region is active and therefore 
Afc > 0. Moreover, the stopping criterion (ED is satisfied after a finite number of 
iterations and the method is regularizing as the error decreases monotonically and 
the sequence } converges to a solution of (ED whenever S goes to zero. 

Theorem 4.2. Suppose that Assumvtions \S. j] and \4^ hold and S > 0. Moreover, 
suppose T > 1/q and k < k^,, where r and A:* are defined in ( EP - Then, Algorithm 
ro generates a sequence xf. such that, for k < k < k^,. 
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(i) the trust-region is active, i.e. Xk > 0 and xf. belongs to Bp{x^); 

(ii) llx; 


fe+i 


— I 


< kc' 


? — tX I 


(Hi) there exists a constant A > 0 such that < A. 

Moreover, 

(iv) the stopping criterion PD) is satisfied after a finite number of iterations and 
the sequence {xf. } converges to a solution of {23P whenever 6 goes to zero. 
Proof, (i)-(ii) By (14.11) and (11.21) we get 

11/ - F{xi) - J[xi){D - 4)11 <S+\\y- F{xi) - J{xi){D - 4)|| 

< 5 + c||a:t -4ll \\y-P{xl)\\ 

< (1 + c\\x^ - 4ll)<5 + c||x^ - x^W 11/ - F{xl)\\. 
Then, at iteration k, condition (El) gives 


\\y^ - Fid) - J{xl){x'‘ - xl)\\ < 


1 + cllx’)' — X* I 


+ c||xt-x||| ||/-n4)IL 


and (14.31) yields (12. 6 |) with 9k = ---r- ^77 > 1. Then, Lemma yields 

l + c(l + r)||xT -x^ll 

A| > 0 and therefore A^ > A| is strictly positive. Further, by a straightforward 
adaptation of m Proposition 4.1], it follows 

114 - - 114+1 -- 4 + Ji4)Pkr. 

and this implies ||x|^^—x’)'|| < ||x| —x'('|| and consequently x|_|_^ G Bp{x^). Repeating 
the above arguments, by induction, we can prove that, for k < k < k^,, condition ( 1 ^ 
holds, Afc > 0 , and 

114 - - 114+1 - - / + JixDpkf, ( 4 . 10 ) 

with 


l + c(l+T)||xt -xfir 

Thus ||Xfc_,_;^ — a;')]] < ||xf — x'('|| and 9k+i > dk for fc < /c < /c*. 

(Hi) Proceeding as in the proof of point (Hi) of Theorem 14.11 just replacing Xk 
with x^, we get that for fc < fc < fc*. A*, < A with 

A < maxi —, —^ I . 

( 70 ; Cmin J 

where uj is obtained replacing y with y^ in (14.7p . 

(iv) Summing up from to fc* — 1, by (|2.13l) and (14.101) it follows 


fc.-i 


{k.,-k)T^S'^ < \\F{xi) - 


.<5||2 


nr < 


%A 


k^k 


m - 1 )/ 


|x^ -X^l 


Thus, fc* is finite for 5 > 0, and convergence of x^^ to a solution of (E as 6 goes to 
0 is shown in pUl Theorem 2.3]. □ 
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5. Numerical results. In this section we report on the performance of the reg¬ 
ularizing trust-region method and make comparisons with the regularizing Levenberg- 
Marquardt method and a standard version of the trust-region method. The test prob¬ 
lems are ill-posed and with noisy data, and arise from the discretization of nonlinear 
Fredholm integral equations of the first kind 

1 

k{t,s,x{s))ds = y{t), tG[0,l]. (5-1) 

The integral equations considered model inverse problems from groundwater hy¬ 
drology and geophysics. Their kernel is of the form 


see dU §3], or 


k{t, s, x(s)) 


log 


[(t-s)^ + (H-x(s)yJ’ 


kit, s, x(s)) 


1 

+ (t - s)2 + x(s)^ ’ 


(5.2) 


(5.3) 


see [m §6]. The interval [0,1] was discretized with n = 64 equidistant grid points 
ti = {i — l)h, h = l/(n — 1), i = 1,... ,n. Function x(s) was approximated from the 
n-dimensional subspace of Hq(0, 1) spanned by standard piecewise linear functions. 
Specifically, we let Sj = (j — l)h, h = l/(n — 1), j = 1,... ,n, and looked for an 
approximation i(s) = X)J=i where 


<('i(s) 

and 


^ if Si < s < S2 ,5 (s) = I < s < s. 

0 otherwise ’ q otherwise 




' if Sj-i <s<Sj, 
if Sj <s< Sj+i, 
0 otherwise 


j = 2,.. .n - 1. 


Finally, the integrals k{ti, s, x{s))ds, 1 < i < n, were approximated by the com¬ 
posite trapezoidal rule on the points sj, 1 < j < n. The resulting discrete problems 
are square nonlinear systems dm with unknown x = (xi,..., x„)^. We observe that 
x{sj) = Xj] thus, the j-th component of x approximates a solution of (I5.1|l at Sj. 

Two problems with kernel ()5.2I1 and two problems with kernel (I5.3|l were consid¬ 
ered and built so that solutions (later denoted as true solutions) are known. Con¬ 
cerning kernel (lO) . the first problem is given in EH p. 46]; it admits as true con¬ 
tinuous solutions the functions xtrue{s) = -I- cs -I- C4 and 

xtrue{s) = 2H — — C3 — C4 where H = 0.2,ci = —0.1,C2 = 

—0.075, c?i = —40,^2 = —60, Pi = 0.4,p2 = 0.67, C3 and C4 are chosen such that 
xtT-ue(O) = xtriie(l) = 0. The second problem was given in [23l p. 835] and it has true 
continuous solutions xtrue{s) = 1.3s(l — s) -I- 0.2 and xtrue{s) = 1.3s(s — 1). 

The third and fourth problems have kernel (|5.3p : the former has solutions xtrue{s) =| 
1 and Xtrue{s) = — 1, s G [0,1], see [131 p. 660], while the latter has the discontinuous 
functions 


Xtrue (- 5 ) 


1 if 0 < S < ^ 

0 if 5 < s < 1 ’ 


Xtrue ( 5 ) 


— 1 if0 <s<^ 
0 if 5 < s < 1 


(5.4) 
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as the true solutions, [H P- 662], 

The nonlinear systems arising from the discretizations of the four problems are 
denoted as PI, P2, P3 and P4 respectively, while x'' £ M" denotes a solution of the 
discretized problems. Given the error level (5, the exact data y was perturbed by 
normally distributed values with mean 0 and variance S using the Matlab function 

randn. 

All procedures were implemented in Matlab and run using Matlab 2014b on an 
Intel Core(TM) i7-4510U 2.6 GHz, 8 GB RAM; the machine precision is ~ 2-10“^®. 
The Jacobian of the nonlinear function F was computed by finite differences. The 
parameter q used in (12.31) and in (12.131) was set equal to 1.1/r and the discrepancy 
principle (El with T = 1.5 was used as the stopping criterion. A maximum number 
of 300 iterations was allowed and a failure was declared when this limit was exceeded. 

In the implementation of the regularizing trust-region method. Step 3 in Algo¬ 
rithm [3TT] was performed setting r] = 5, 7 = Then, in Step 2 the trust-region radius 
was updated as follows 

Ao = /ioll^l)||, Mo = 10-\ (5.5) 

Afc+i =/rfe+i||F(xfc+i)||, = J iiqk<q, ^ 

I 2yk if Qk > vq, 


with q^ 


\\J{xi)pk + F{xi)\\ 

mxi )\\ 


and zz = 1.1. The maximum and minimum values for 


Ak were set to A^ax = 10"* and Amin = 10“^^. This updating strategy turned out 
to be efficient in practice and was based on the following considerations. Glearly, A^ 
is cheaper to compute than the upper bound in (13.71) and preserves the property of 
converging to zero as <5 tends to zero and a solution of problem El is approached. 
Further, Ak is adjusted taking into account the g-condition and by monitoring the 
value qk', therefore, if the g-condition was not satisfied at the last computed iterate 

it is reasonable to take a smaller radius than in the case where the g-condition 
was fulfilled. 

The computation of the parameter Xk was performed applying Newton method 
to the equation 


Vz(A) 


lb(-^)ll Afe 


(5.7) 


and each Newton iteration requires the Gholesky factorization of a shifted matrix of 
the form Bk + XI [3]. Typically high accuracy in the solution of the above scalar 
equations is not needed mm and this fact was experimentally verified also for our 
test problems. Hence, after extensive numerical experience, we decided to terminated 
the Newton process as soon as |Afc — ||p(A)||| < 10“^A^. 

In our implementation of the standard trust-region method, we chose the trust- 
region radius accordingly to technicalities well-known in literatures, see e.g. [31 §6.1] 
and [19l §11.2]. In particular, we set Aq = 1, 


A 


fc-i-i — 


M 

4 


Afe, 

Lirin'[2A/i;, Amax} ? 


if Pk < j, 

if i < Pfc < I, 
otherwise. 


with Amax = 10^ and chose Amin = 10 as the minimum values for A^,. 
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Problem 




RTR 



RLM 


Xo 

it 

l|F|| 

nf 

cf 

ei 

ei 

ei 

Gt 

PI 

Oe 

43 

1.3e-4 

44 

5 

5.5e-3 

5.5e-3 

4.5e-3 

4.5e-3 


-0.5 e 

63 

1.2e-4 

71 

5 

3.2e-2 

7.9e-2 

3.0e-2 

7.1e-2 


—1 e 

82 

1.4e-4 

94 

4 

3.4e-2 

8.4e-2 

4.0e-2 

7.2e-2 


-2e 

115 

1.5e-4 

138 

4 

3.4e-2 

8.6e-2 

2.9e-2 

6.1e-2 

P2 

Oe 

54 

1.2e-4 

55 

5 

7.4e-3 

7.4e-3 

* 

* 


0.5 e 

56 

1.4e-4 

59 

5 

l.le-2 

1.3e-2 

* 

* 


1 e 

73 

1.4e-4 

84 

4 

l.Oe-2 

1.3e-2 

7.3e-3 

8.3e-3 


2e 

118 

1.4e-4 

138 

4 

9.3e-3 

l.le-2 

4.8e-3 

4.8e-3 

P3 

a:o(1.25) 

35 

1.4e-4 

36 

3 

1.2e-2 

1.2e-2 

3.1e-3 

3.1e-3 


a;o(l-5) 

43 

1.4e-4 

44 

3 

5.1e-2 

5.1e-2 

6.2e-2 

6.2e-2 


a:o(1.75) 

45 

1.3e-4 

46 

3 

3.2e-l 

3.2e-l 

3.1e-l 

3.1e-l 


a:o(2) 

65 

1.4e-4 

71 

3 

4.6e-l 

4.6e-l 

3.8e-l 

3.8e-l 

P4 

2 : 0 ( 1 ,1) 

68 

1.5e-4 

82 

3 

4.8e-l 

4.8e-l 

* 

* 


a;o(0.5,0) 

64 

1.5e-4 

75 

3 

4.9e-l 

4.9e-l 

4.7e-l 

4.7e-l 


2;o(1.5,1) 

69 

1.5e-4 

78 

3 

5.1e-l 

5.1e-l 

4.8e-l 

4.8e-l 


a;o(1.5,0) 

68 

1.5e-4 

78 

4 

5.2e-l 

7.1e-l 

5.1e-l 

6.3e-l 


Table 5.1 

Results obtained by the regularizing trust-region method and the regularizing Levenberg- 
Marquardt method with noise 6 = 10 “^ and varying initial guesses. 


Finally the Levenberg-Marquardt approach was implemented imposing condition 
(lO) and solving (12.111) by Newton method. In order to find an accurate solution for 
(12.311 it was necessary to use a tighter tolerance, equal to 10“®, than that used in the 
trust-region algorithm. 

Our experiments were made varying the noise level 5 on the data . Tables [ 53 ] 
and 15.21 display the results obtained by the regularizing trust-region algorithm with 
noise 5 = 10“"^ and 5 = 10“^ respectively. Runs for four different initial guesses 
xq are reported in the tables. For problems PI and P2 the initial guesses are xq = 
Oe, —0.5e, —e, —2e and xq = Oe, 0.5e,e, 2e respectively, where e denotes the vector 
e = (1,...,!)^. For problem P3 the initial guess was chosen as the vector xo(a) 
with j-th component given by {xo{a))j = gaisj) for j = l,...,n, where ga{s) = 
(—4a -I- 4)s^ -I- (4a — 4)s -|- 1, and sj being the grid points in [0,1]. We have used 
the following values of a, a = 1.25,1.5,1.75,2. For problem P4 the initial guess 
xo{P,x) has components {xo{(3,x))j = j = 1 ,... ,n with g/^^^ = jj - 

and (/3,x) = (1,1), (0.5,0), (1.5,1), (1.5,0). In the tables we report: the initial guesses 
(for increasing distance from the true solutions) the number of iterations it performed; 
the final norm of function F\ the number of function evaluations nf performed; the 
rounded average number cf of Cholesky factorizations per iteration. To assess the 
quality of the results obtained, we measured the distance between the final iterate 
cCfc. and the true solution approached; in particular ej = max2<j<n-i \xtrue{sj) — 
^he maximum absolute value of the difference between the components 
associated to internal points Sj £ (0,1), while bt = maxi<j<„ \xtrue{sj) — (a^fc»)j| is 
the maximum absolute value of the difference between the components associated to 
points Sj including the end-points of the interval [0,1]. The symbol indicates that 
either the procedure failed to satisfy the discrepancy principle within the prescribed 
maximum number of iteration, or the final xf., was not an approximation of one of 
the true solutions described above. 
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Problem 




RTR 



RLM 


Xq 

it 

l|A|| 

nf 

cf 

ei 

Ct 

ei 

Ct 

PI 

Oe 

20 

1.5e-2 

21 

6 

1.9e-2 

1.9e-2 

1.8e-2 

1.8e-2 


-0.5 e 

29 

l.Oe-2 

30 

6 

2.2e-2 

3.1e-l 

2.1e-2 

3.1e-l 


—1 e 

35 

1.4e-2 

36 

5 

3.6e-2 

6.1e-l 

3.3e-2 

6.1e-l 


-2e 

40 

1.3e-2 

41 

5 

4.9e-2 

1.2e-P0 

4.5e-2 

1.2e-P0 

P2 

Oe 

30 

1.4e-2 

31 

5 

6.9e-3 

1.3e-2 

* 

* 


0.5 e 

25 

1.4e-2 

26 

5 

1.7e-2 

2.1e-l 

* 

* 


1 e 

29 

1.4e-2 

30 

5 

3.8e-2 

5.4e-l 

1.3e-l 

5.2e-l 


2e 

37 

1.4e-2 

39 

5 

5.5e-2 

1.2e-P0 

2.2e-l 

l.le-PO 

P3 

*0(1.25) 

15 

1.2e-2 

16 

4 

1.5e-l 

1.5e-l 

1.5e-l 

1.5e-l 


* 0 ( 1 . 5 ) 

17 

1.4e-2 

18 

4 

3.2e-l 

3.2e-l 

3.2e-l 

3.2e-l 


* 0 ( 1 . 75 ) 

19 

1.4e-2 

20 

4 

5.0e-l 

5.0e-l 

5.1e-l 

5.1e-l 


*o(2) 

22 

1.5e-2 

23 

4 

6.9e-l 

6.9e-l 

7.0e-l 

7.0e-l 

P4 

* 0 ( 1 ,1) 

17 

1.4e-2 

18 

5 

5.7e-l 

5.7e-l 

5.4e-l 

5.4e-l 


* 0 ( 0 . 5 ,0) 

20 

1.3e-2 

21 

4 

5.5e-l 

5.5e-l 

* 

* 


* 0 ( 1 . 5 ,1) 

22 

1.4e-2 

23 

4 

5.1e-l 

5.1e-l 

5.0e-l 

5.0e-l 


* 0 ( 1 . 5 ,0) 

26 

1.5e-2 

27 

4 

5.2e-l 

8.8e-l 

* 

* 


Table 5.2 

Results obtained by the regularizing trust-region method and the regularizing Levenberg- 
Marquardt method with noise 5 = 10 “^ and varying initial guesses. 


Tables 15.11 and 15.21 show that the regularizing trust-region method solves all the 
tests. By Step 3 of our Algorithm 13.11 the difference between the number of function 
evaluations and the number of trust-region iterations, if greater than one, indicates 
the number of trial iterates that were rejected because a sufficient reduction on $ was 
not achieved. We observe that in 20 out of 32 runs, all the iterates generated were 
accepted; this occurrence seems to indicate that the trust-region updating rule works 
well in practice. 

Further insight on the trust-region updating rule (I5.5|) - (I5.6D can be gained an¬ 
alyzing the regularizing properties of the implemented trust-region strategy. First, 
we verified numerically that, though not explicitly enforced, the g-condition is sat¬ 
isfied in most of the iterations. As an illustrative example, we consider problem P2 
with S = 10“'^ and xq = Oe and, in the left plot in Figure ISTl we display the values 
qk = at the trust-region iterations, marked by an asterisk, and the 

value q = 1.1/r Ri 0.733 fixed in our experiments, depicted by a solid line. We ob¬ 
serve that, even if we have not imposed the g-condition, it is satisfied at most of the 
iterations. The plot on the right of Figure 15.11 shows a monotone decay of the error 
between xf. and x^ through the iterations, which results to be in accordance with the 
theoretical results in Theorem 14.21 The regularizing properties of the implemented 
trust-region scheme are also shown in Figure [52] where, for each test problem we plot 
the error ~ 3;^ll for decreasing noise levels; it is evident that, in accordance with 
theory, the error decays as the noise level decreases. 

Let now compare the regularizing trust-region and Levenberg-Marquardt proce¬ 
dures. On runs successful for both methods, the two methods provide solutions of 
similar accuracy and such an accuracy increases with the vicinity of the initial guess 
to the true solution; as an example Figure 15.31 shows the solutions computed by the 
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Regularizing trust-region applied to P2, xq = Oe, S = 10 values = 

(marked by an asterisk) and value of q = l.l/r (solid line) versus the iterations 

versus the iterations (on the right). 


Fig. 5.1. 

II J(^fc)pfc+-p'(^fc 

(on the left); semilog plot of the error 


4-xt| 


two methods for problems PI and P3 for 5 = 10“^. On the other hand, for large noise 
5 and initial guesses farther from the true solution, for both methods the accuracy at 
the endpoints of the interval [0,1] may deteriorate; for this occurrence we refer to Ta- 
ble 15.21 and runs on problems PI and P2. Concerning failures, in 7 runs out of 32 the 
Levenberg-Marquardt algorithm does not act as a regularizing method as the gener¬ 
ated sequence approaches a solution of the noisy problem. In Figure [SH] we illustrate 
two unsuccessful runs of the Levenberg-Marquardt method; approximated solution 
computed by the regularizing trust-region and Levenberg-Marquardt procedures are 
shown for runs on problems P2 and P4. 

The overall experience on the Levenberg-Marquardt algorithm seems to indicate 
that the use of the g-condition is more flexible than condition (12.31) and provides 
stronger regularizing properties. In order to support this claim, in Figure 15.51 we 
report four solution approximations computed by the Levenberg-Marquardt algorithm 
for varying values of q, i.e. q = 0.67,0.70,0.73,0.87. It is evident that the method 
is highly sensitive to the choice of the parameter q and the quality of the solution 
approximation does not steadily improves as q increases. 

We conclude this section considering the standard trust-region strategy. It is well 
known that the standard updating rule promotes the use of inactive trust-regions, 
at least in the late stage of the procedure. Clearly, this can adversely affect the so¬ 
lution of our test problems and our experiments confirmed this fact. In particular, 
for 6 = 10“^ and problems PI and P2, the sequences computed by the standard 
trust-region method approach solutions of the noisy problem. The same behaviour 
occurs in most of the runs with PI and P2 and noise level 5 = 10“^. Conversely, 
the approximations provided by the regularizing trust-region procedure are accurate 
approximations of true solutions in all the tests. Moreover, the approximations com¬ 
puted by the standard trust-region applied to problems P3 and P4 are less accurate 
than those computed by the regularizing trust-region although they do not show the 
strong oscillatory behavior arising in problems PI and P2. In problem P4, this behav¬ 
ior is evident when the second, third and fourth starting guesses are used, while the 
approximation computed starting from the first initial guess is as accurate as the one 
computed by the regularizing trust-region. This good result of the standard trust- 
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Fig. 5.2. Regularizing trust-region applied to PI, xq = Oe (top left), P2, xq = Oe (top right), 
P3, xq = X(){a) = a;o(1.25) (lower left) and to Pf, xq = xq{/3,x) = aio (0-5 iO ) (lower right): log plot 
of the error ~ 3 ;t|| versus the noise S. 


region on problem P4 with Xq = a:o(l,l) is due to the fact that the trust-region is 
active in all iterations and therefore a regularizing behaviour is implicitely provided. 
As an example in Figure 15.61 we compare some solution approximations computed 
by the regularizing trust-region (left) and by the standard trust-region (right) with 
S = 10“^ applied to problem PI (figures (a)-(b)), P2 (figures (c)-(d)), P3 (hgures 
(e)-(f)) and P4 (figures (g)-(h)). 


6. Conclusions. We have presented a trust-region method for nonlinear ill- 
posed systems, possibly with noisy data, where the regularizing behaviour is guaran¬ 
teed by a suitable choice of the trust-region radius. The proposed approach shares the 
same local convergence properties as the regularizing Levenberg-Marquardt method 
proposed by Hanke in m but it is more likely to satisfy th discrepancy principle 
irrespective of the closeness of the initial guess to a solution of (HH). The numerical 
experience presented confirms the effectiveness of the trust-region radius adopted and 
the regularizing properties of the resulting trust-region method. It also enlights that 
the new approach is less sensitive than the regularizing Levenberg-Marquardt method 
to the choice of the parameter q in (I2.3p and (I2.13|l . Finally, as expected the numerical 
results show that the solution of the noisy problems may be misinterpreted by the 
standard trust-region method. 
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plot of the true and the computed solution plot of the true and the computed solution 




plot of the true and the computed solution plot of the true and the computed solution 




Fig. 5.3. Regularizing trust-region (left) and regularizing Levenberg-Marquardt (right), true 
solution (solid line) and approximate solutions (dotted line). Upper part: PI, 6 = 10~^, a:o = Oe; 
lower part: P3, 5 = 10~^, a:o = a^o(Q:) = xo(1.25). 
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Fig. 5.6. True solution (solid line) and approximate solutions (dotted line) computed by the 
regularizing trust-region method (on the left) and the standard trust-region method (on the right). 
(a)-(b) problem PI, S = 10“^, 3:0 = Oe; (c)-(d) problem P2, S = 10“^, 3:0 = Oe; (e)-(f) problem P3, 
































































